Portable device and method of controlling therefor

ABSTRACT

A portable device is disclosed. The portable device according to one embodiment includes a camera unit configured to capture an image in front of the portable device, a display unit configured to display a virtual image, and a processor configured to control the camera unit and the display unit, the processor further configured to detect a marker object from the image, display the virtual image corresponding to the marker object based on a position of the marker object when the marker object is detected, detect a position change of the marker object in the image, move the virtual image according to the position change when the position change is detected and obtain a first moving speed of the virtual image or a second moving speed of the marker object, when the first moving speed or the second moving speed is faster than a first reference speed, lower the first moving speed to less than the first reference speed.

Pursuant to 35 U.S.C. §119(a), this application claims the benefit ofthe Korean Patent Application No. 10-2013-0105125, filed on Sep. 2,2013, which is hereby incorporated by reference as if fully set forthherein.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present specification relates to a portable device, and moreparticularly, to a portable device providing an augmented reality (AR)service by displaying a virtual image for a real object and a method ofcontrolling therefor.

2. Discussion of the Related Art

As a technology has been developed, a portable device changes itself asa smart device capable of providing various services to a user.Recently, the portable device is utilized as a device providing anaugmented reality (AR) service, which corresponds to a service showing areal image and a virtual image together in a manner of mixing bothimages.

The portable device provides the AR service using a method that avirtual image corresponding to a real object, which is detected from anearby image, is displayed in a manner of being overlapped to the realobject.

In case of providing the AR service with the aforementioned displaymethod, if the real object moves fast, the virtual image moves fast aswell. Hence, it is difficult for a user to observe the virtual image orto obtain information included in the virtual image.

SUMMARY OF THE INVENTION

Accordingly, the present specification is directed to an apparatus andmethod thereof that substantially obviate one or more problems due tolimitations and disadvantages of the related art.

In providing an AR service, an object of the present specification is toprovide a portable device changing a method of displaying a virtualimage according to whether the virtual image moves and a moving speed ofthe virtual image and a method of controlling therefor.

Additional advantages, objects, and features of the invention will beset forth in part in the description which follows and in part willbecome apparent to those having ordinary skill in the art uponexamination of the following or may be learned from practice of theinvention. The objectives and other advantages of the invention may berealized and attained by the structure particularly pointed out in thewritten description and claims hereof as well as the appended drawings.

To achieve these objects and other advantages and in accordance with thepurpose of the invention, as embodied and broadly described herein, aportable device according to one embodiment includes a camera unitconfigured to capture an image in front of the portable device, adisplay unit configured to display a virtual image, and a processorconfigured to control the camera unit and the display unit, theprocessor further configured to detect a marker object from the image,display the virtual image corresponding to the marker object based on aposition of the marker object when the marker object is detected, detecta position change of the marker object, move the virtual image accordingto the position change when the position change is detected, and obtaina first moving speed of the virtual image or a second moving speed ofthe marker object, when the first moving speed or the second movingspeed is faster than a first reference speed, lower the first movingspeed to less than the first reference speed.

To achieve these objects and other advantages and in accordance with thepurpose of the invention, as embodied and broadly described herein, amethod of controlling a portable device includes the steps of capturingan image in front of the portable device, detecting a marker object fromthe image, when the marker object is detected, obtaining an informationon the virtual image corresponding to the marker object and displayingthe virtual image based on a position of the marker object, detecting aposition change of the marker object, when the position change isdetected, moving the virtual image according to the position change andobtaining a moving speed of the virtual image or a moving speed of themarker object, and when the moving speed of the virtual image or themoving speed of the marker object is faster than a first referencespeed, lowering the moving speed of the virtual image less than thefirst reference speed.

According to the present specification, a portable device according toone embodiment can provide a user-friendly AR service. Morespecifically, in providing an AR service, the portable device canprovide a virtual image readable by a user irrespective of a movingspeed of a marker object in a manner of controlling a moving speed ofthe virtual image moving in accordance with a position change of themarker object. And, in case of controlling the moving speed of thevirtual image, the portable device can further enhance readability ofthe virtual image in a manner of changing a method of displaying thevirtual image. And, the portable device can determine whether thevirtual image is continuously displayed according to the moving speed ofthe detected virtual image or a size of the marker object and then canstop displaying an unnecessary virtual image, thereby reducing powerconsumption of the portable device.

It is to be understood that both the foregoing general description andthe following detailed description of the present specification areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 is a block diagram of a portable device according to oneembodiment;

FIGS. 2A-2B are diagrams of a portable device capturing an image infront of a device and displaying a virtual image according to oneembodiment;

FIGS. 3A-3C are diagrams of a portable device displaying a virtual imagefor a plurality of detected marker images according to one embodiment;

FIGS. 4A-4B are diagrams of a case that a position change of a markerobject is detected according to one embodiment;

FIGS. 5A-5D are diagrams of a portable device displaying a virtual imagein accordance with a moving speed of the virtual image according to oneembodiment;

FIGS. 6A-6B are diagrams of a portable device displaying a virtual imagein accordance with duration of which the marker object is detected or asize of which the marker object is detected according to one embodiment;

FIGS. 7A-7D are diagrams of a portable device displaying a virtual imagewith a changed displaying method in case of lowering a moving speed ofthe virtual image according to one embodiment;

FIG. 8 is a flowchart of a method of controlling a portable deviceaccording to one embodiment;

FIG. 9 is a flowchart of a method of controlling a portable deviceaccording to a different embodiment.

DETAILED DESCRIPTION OF THE INVENTION

Although terminologies used in the present specification are selectedfrom general terminologies used currently and widely in consideration offunctions, they may be changed in accordance with intentions oftechnicians engaged in the corresponding fields, customs, advents of newtechnologies and the like. Occasionally, some terminologies may bearbitrarily selected by the applicant(s). In this case, the meanings ofthe arbitrarily selected terminologies shall be described in thecorresponding part of the detailed description of the specification.Therefore, terminologies used in the present specification need to beconstrued based on the substantial meanings of the correspondingterminologies and the overall matters disclosed in the presentspecification rather than construed as simple names of theterminologies.

Moreover, while the embodiments have been concretely described withreference to the attached diagrams and the contents written on thediagrams, a scope of claims may be non-restricted or non-limited to theembodiments.

The present specification relates to a portable device. In the presentspecification, the portable device corresponds to a portable electronicdevice and may mean an electronic device capable of capturing an imagein front of the device, detecting a marker object from the capturedimage, and displaying a virtual image related to the detected markerobject. For instance, the portable device may correspond to such variouselectronic devices as a smartphone, a smart pad, a personal digitalassistant (PDA), a tablet PC, a smart table, a head mounted display(HMD), or the like.

In the present specification, a marker object may mean an objectincluding a marker to indicate whether a related virtual image exists orwhether a virtual image is displayed. In this case, the marker maycorrespond to a promised pattern or a form of a code equipped on asurface of a random object. Or, the marker may correspond to a randomobject itself. These kinds of all optical identifiers can be called amarker. And, in the present specification, a virtual image correspondsto an image consisted of a text and a graphic and indicates an imageconfigured to provide an augmented reality information related to amarker object to a user. For instance, if the marker object correspondsto a bus, the virtual image may correspond to an image configured toprovide such augmented reality information as information on a bus line,an allocation interval, or a position of the next bus.

FIG. 1 is a block diagram of a portable device according to oneembodiment.

Referring to FIG. 1, the portable device 100 includes a camera unit 110,a display unit 130, a sensor unit 150, and a processor 170.

The camera unit 110 can capture an image in front of a device. Morespecifically, the camera unit 110 captures an image in front of a devicewithin an angle of view and can deliver the captured image to theprocessor 170. In this case, the angle of view indicates an anglecapable of taking an image via a lens of a camera. In this case, thecaptured image may correspond to a form of a picture, a video, a stillframe. And, the captured image can include at least one marker object.

The display unit 130 can display a virtual image. More specifically, thedisplay unit 130 can display a virtual image based on a control commandof the processor 170.

The display unit 130 can be equipped with a non-transparent or atransparent display panel. In this case, the transparent display panelmay include a translucent display panel. For instance, thenon-transparent display panel may correspond to a liquid crystal display(LCD), a plasma display panel (PDP), a light-emitting diode display(LED), an organic light-emitting diode display (PLED), or the like. Forinstance, the transparent display panel may correspond to an opticalsee-through display panel.

The sensor unit 150 senses a user input or environment of a device usingat least one sensing means installed in the device and can deliver asensed result to the processor. The at least one sensing means mayinclude such a sensing means as gravity sensor, a terrestrial magnetismsensor, a motion sensor, a gyroscope sensor, an acceleration sensor, aninfrared sensor, a tilt (inclination) sensor, a brightness sensor, analtitude sensor, a smell sensor, a temperature sensor, a depth sensor, apressure sensor, a bending sensor, an audio sensor, a video sensor, aGPS (global positioning system) sensor, a touch sensor, and the like.The aforementioned sensors may be included in the device as a separateelement or may be included in the device in a manner of being combinedto at least one element.

A storage unit can store such a various digital data as a video, anaudio, a picture, an application, and the like. The storage unit can beinstalled in the inside or the outside of the portable device 100. Forinstance, an internal storage unit can be installed in the inside of theportable device using a storing means such as a flash memory, a harddisk drive (HDD), a solid state drive (SSD). As a different example, anexternal storage unit can be installed in the outside of the portabledevice 100 using a server on the internet represented as a cloud. Theexternal storage unit using the cloud can provide a data existing in aposition physically different from each other in a manner of integratingthe data with a virtualizing technique.

The communication unit can transceive a data with an external in amanner of performing a communication using various protocols. Thecommunication unit accesses an external network by wired or wireless andcan transceive a digital/analog data with the network.

The processor 170 executes various applications and can process a dataof the internal of the device. And, the processor 170 controls each ofthe units of the aforementioned portable device 100 and can control datatransmission/reception between units.

The processor of the present specification detects a marker object froma captured image and can display a virtual image corresponding to thedetected marker object based on a position of the marker object. And,the processor detects a position change of the marker object, moves thevirtual image according to the position change, and can obtain a firstmoving speed of the virtual image. And, when the first moving speed isfaster than a first reference speed, the processor can slow down thefirst moving speed to make the first moving speed less than a firstreference speed. Detailed explanation on the processor shall bedescribed in the following description with reference to each drawing.

First of all, FIG. 1 is a block diagram of a portable device 100according to one embodiment. Blocks represented in a manner of beingseparated indicate the logically distinguished elements of the device.Hence, the elements of the aforementioned device can be equipped with asingle chip or a plurality of chips according to the design of thedevice. And, the portable device 100 depicted in FIG. 1 is just oneembodiment. All elements included in FIG. 1 are not necessary to performembodiments of the present specification. In the following description,elements required for the embodiment of the present specification andoperations of the elements are explained in detail.

FIG. 2 is a diagram of a portable device capturing an image in front ofthe portable device and displaying a virtual image according to oneembodiment.

FIG. 2 a is an embodiment of the portable device including a displayunit equipped with a non-transparent display panel and FIG. 2 b is anembodiment of the portable device including a display unit equipped witha transparent display panel.

Referring to FIG. 2 a and FIG. 2 b, the portable device 100 can capturean image 20 in front of the device within an angle of view 10 area usinga camera unit. In this case, the camera unit can capture the image 20 infront of the device contiguously or with a specific time interval. Forinstance, the camera unit contiguously captures the image 20 in front ofthe device and can deliver the captured image 20 of a video form to theprocessor. As a different example, the camera unit captures the image 20in front of the device with a specific time interval and can deliver thecaptured image 20 of a still frame form to the processor.

And, the portable device 100 can detect a marker object 30 using theprocessor. For instance, the processor detects the marker object 30 fromthe captured image 20 captured by the camera unit or can detect themarker objects 30 from a data corresponding to the image 20 displayed inthe display unit. In this case, the processor can detect the markerobject 30 in a manner of applying a pre-stored detecting algorithm.

And, the portable device 100 can obtain information (hereinafter alsocalled virtual image information) on a virtual image 40 related to themarker object 30 using the processor. For instance, the processordirectly obtains the virtual image information from an internal storageunit or can obtain the virtual image information from an externalstorage unit via the communication unit. In this case, the externalstorage unit can provide the virtual image information existing in aposition physically different from each other to the portable device viaa wired/wireless communication network in a manner of integrating thevirtual image information with a virtualization technique.

In this case, the virtual image information can include at least one ofaugmented reality information related to the marker object, informationon a display mode (display mode information), information on a graphicuser interface format (graphic user interface format information).

In this case, the display mode is a mode to determine a quantity of theaugmented reality information included in a display object. The displaymode can be classified into a general display mode and a simple displaymode. The general display mode is the display mode configured to includeall augmented reality information included in the virtual imageinformation in the display object. The simple display mode is thedisplay mode configured to include selected augmented realityinformation in the display object only among the augmented realityinformation included in the virtual image information.

In this case, the graphic user interface format means a user interfaceformat to display a virtual image in the display unit. The graphic userinterface format according to one embodiment can include at least one ofa display position, a display size, a display shape, a display color ofthe virtual image, or a display transparency of the virtual image.

And, the portable device 100 can display the virtual image 40 based on aposition of the marker object 30 using the processor. For instance, asdepicted in FIG. 2 a, the processor can display the virtual image 40 inthe top of the marker object 30 corresponding to the virtual image. As adifferent example, the processor can display the virtual image 40 insuch various positions as the bottom, the left, the right, or the likeof the marker object 30 corresponding to the virtual image 40. As afurther different example, the processor can display the virtual image40 in a manner of overlapping the marker object 30 corresponding to thevirtual image with the virtual image 40.

As depicted in FIG. 2 a, in case of a display unit 130 equipped with anon-transparent panel, the portable device 100 can display a capturedimage 20 and the virtual image 40 together. As depicted in FIG. 2 b, incase of the display unit equipped with a transparent display panel, theportable device 100 can display the virtual image 40 in a manner thatthe virtual image 40 overlaps a real environment of which a user seeswithout displaying the captured image 20.

FIG. 3 is a diagram of a portable device displaying a virtual image fora plurality of detected marker images according to one embodiment.

More specifically, FIG. 3 a is an embodiment of the portable devicedisplaying a virtual image for all of a plurality of detected markerobjects, FIG. 3 b is an embodiment of the portable device displaying avirtual image for a selected marker object among a plurality of thedetected marker objects, and FIG. 3 c is an embodiment of the portabledevice displaying a virtual image for a marker object positioned at auser gaze among a plurality of the detected marker objects.

In FIG. 3 a to FIG. 3 c, since detail contents of obtaining the virtualimage information and displaying the virtual image based on a positionof a marker object are identical to the contents mentioned earlier inFIG. 2, detail contents are omitted.

First of all, referring to FIG. 3 a, when a plurality of marker objects(30-1, 30-2, 30-3, and 30-4) are detected, the portable device canobtain virtual image information related to each of a plurality of themarker objects (30-1, 30-2, 30-3, and 30-4). The portable device candisplay each of virtual images (40-1, 40-2, 40-3, and 40-4) based onpositions of the corresponding marker objects (30-1, 30-2, 30-3, and30-4) using the obtained virtual image information.

Referring to FIG. 3 b, when a plurality of the marker objects (30-1,30-2, 30-3, and 30-4) are detected, the portable device selects specificmarker objects (30-1 and 30-2) among a plurality of the marker objects(30-1, 30-2, 30-3, and 30-4) according to a predetermined selectioncondition and can obtain virtual image information related to theselected specific marker objects (30-1 and 30-2). In this case, theselection condition is a condition to select the specific marker objectdisplaying a related virtual image. The selection condition may includeinformation on an attribute and a type of a marker object. The selectioncondition may be a condition configured based on a user input. And, theselection condition may correspond to a condition stored in advance inthe portable device.

For instance, as depicted in FIG. 3 b, if mobility of a marker object isconfigured as the selection condition, the portable device selectsspecific marker objects (30-1 and 30-2) having mobility among aplurality of the marker objects (30-1, 30-2, 30-3, and 30-4) and canobtain virtual image information related to the selected specific markerobjects (30-1 and 30-2). And, the portable device can display each ofthe virtual images (40-1 and 40-2) based on the positions of thecorresponding specific marker objects (30-1 and 30-2) using the obtainedvirtual image information.

Referring to FIG. 3 c, in case that a plurality of the marker objects(30-1, 30-2, 30-3, and 30-4) are detected, the portable device senses auser gaze using the sensor unit 150 and can detect a gaze location wherethe user's gaze is on, and can select at least one marker object fromthe plurality of marker objects (30-1, 30-2, 30-3, and 30-4) based onthe gaze location.

For instance, as depicted in FIG. 3 c, the portable device can selectone marker object (30-1) positioned at the sensed gaze location orpositioned at an area adjacent to the sensed gaze location from aplurality of the detected marker objects (30-1, 30-2, 30-3, and 30-4).And, the portable device obtains virtual image information on theselected marker objects (30-1) and can display a virtual image (40-1)based on a position of the selected marker object (30-1) using theobtained virtual image information.

As a different example, the portable device can select a plurality ofmarker objects (30-1 and 30-3) positioned within a predetermineddistance from the sensed gaze location from a plurality of detectedmarker objects (30-1, 30-2, 30-3, and 30-4). And, the portable deviceobtains virtual image information on the selected marker objects (30-1and 30-3) and can display each of virtual images (40-1 and 40-3) basedon the positions of the selected marker objects (30-1 and 30-3) usingthe obtained virtual image information.

FIG. 4 is a diagram of a case that a position change of a marker objectis detected according to one embodiment. The position change of themarker can occur due to an absolute position change of the marker objector a relative position change between the portable device and the markerobject.

FIG. 4 a is an embodiment for the absolute position change of the markerobject and FIG. 4 b is an embodiment for the relative position change ofthe marker object. In this case, the processor can detect the positionchange of the marker object 30 in a manner of applying a pre-storeddetecting algorithm or a tracking algorithm. And, the portable devicecan sense a position and a direction of the marker object using thesensor unit. The portable device can determine whether the positionchange corresponds to the absolute position change or the relativeposition change using a sensed result and can perform a data processingthat corresponds to each case.

Referring to FIG. 4 a, since the marker object 30 is moving and theportable device 100 is at a standstill, the portable device 100 candetect the absolute position change of the marker object 30.

For instance, the portable device 100 can detect the absolute positionchange of the marker object 30 from an image captured by the camera unitusing the processor. As a different example, the portable device 100 candetect the absolute position change of the marker object 30 from a datacorresponding to an image displayed in the display unit 130 using theprocessor. In this case, the marker object 30 may correspond to such amovable object as a bus, a subway, and the like.

Referring to FIG. 4 b, since the marker object 30 is at a standstill andthe portable device 100 is moving, the portable device 100 can detectthe relative position change of the marker object 30.

For instance, the portable device 100 can detect the relative positionchange of the marker object 30 from an image captured by the camera unitusing the processor. As a different example, the portable device candetect the relative position change of the marker object 30 from a datacorresponding to an image displayed in the display unit 130 using theprocessor. In this case, the marker object 30 may correspond to such anobject fixed on a GPS as a building, a bus stop information board, andthe like.

In this case, the portable device 100 can be moved based on a usercarrying the device or the user wearing the device. And, the portabledevice can be moved based on a movement of such a means oftransportation as a bus and the like on which the user carrying thedevice or the user wearing the device is getting.

As mentioned in the foregoing description, when a position change of themarker object 30 is detected, the portable device 100 moves a virtualimage 40 according to the position change of the marker object 30 andcan obtain a moving speed (also called a first moving speed) of thevirtual image 40 using the processor. In this case, the processor canobtain the moving speed of the virtual image 40 from an image displayedin the display unit 130.

And, the processor can further obtain a moving speed (also called asecond moving speed) of the marker object 30. In this case, theprocessor can obtain the moving speed of the marker object 30 from theimage captured by the camera unit or the image displayed in the displayunit 130. In this case, the processor may obtain the moving speed of themarker object 30 only when the absolute position change of the markerobject 30 is detected.

And, when the position change of the marker object is detected, theprocessor can further obtain acceleration (also called a first movingacceleration) of a virtual image or acceleration of a marker object(also called a second moving acceleration).

FIG. 5 is a diagram of a portable device displaying a virtual image inaccordance with a moving speed of the virtual image according to oneembodiment. FIG. 5 shows virtual images displayed in the display unit oneach time (t1, t2, t3, and t4) and an interval between each of the times(t1, t2, t3, and t4) is identical to each other.

FIG. 5 a is an embodiment of the portable device displaying a virtualimage when a moving speed of the virtual image is slower than a firstreference speed, FIG. 5 b is an embodiment of the portable devicedisplaying a virtual image when a moving speed of the virtual image isfaster than the first reference speed and slower than a second referencespeed, FIG. 5 c is an embodiment of the portable device displaying avirtual image when a moving speed of the virtual image is faster thanthe second reference speed, and FIG. 5 d is an embodiment of theportable device displaying a virtual image when a moving speed of thevirtual image is faster than the first reference speed but accelerationof the virtual image is negative acceleration.

Referring to FIG. 5 a, the moving speed of the virtual image 40 on eachtime (t1, t2, and t3) is slower than the first reference speed. In thiscase, the portable device moves the virtual image 40 according to aposition change of a marker object 30. The portable device cancontinuously display the virtual image 40 in a position adjacent to themarker object 30.

In this case, the first reference speed is a predetermined speed andindicates a minimum speed required to read a moving virtual image. Inparticular, if the moving speed of the virtual image 40 is slower thanthe first reference speed, a user can sufficiently read the virtualimage 40. Yet, if the moving speed of the virtual image 40 is fasterthan the first reference speed, the user cannot sufficiently read thevirtual image 40.

The first reference speed can be configured based on at least one of asize ratio of a display area and a virtual image, quantity of augmentedreality information included in a virtual image, or resolution of adisplay unit. For instance, the bigger the size of the display areacompared to the size of the display unit, the less quantity of theaugmented reality information included in the virtual image, and thehigher the resolution of the display unit, the greater the firstreference speed value increases. The first reference speed can be set tovarious values according to a user configuration and can be set withvarious ways.

Referring to FIG. 5 b, the moving speed of the virtual image 40 on eachtime (t1, t2, and t3) is faster than the first reference speed andslower than the second reference speed. In this case, the portabledevice can lower the moving speed of the virtual image 40 to less thanthe first reference speed. And, when the moving speed of the virtualimage 40 is lowered less than the first reference speed, the portabledevice changes a graphic user interface format and can display thevirtual image in accordance with the changed graphic user interface.Regarding this, it shall be described in detail in FIG. 7.

In this case, the second reference speed is a predetermined speed andindicates a maximum speed required to read a moving virtual image 40. Inparticular, if the moving speed of the virtual image 40 is faster thanthe second reference speed, it is impossible or unnecessary for a userto sufficiently read the virtual image 40. Hence, the portable devicecan stop displaying the virtual image 40. The second reference speed isfaster than the first reference speed.

The second reference speed can be configured based on at least one of asize ratio of a display area and a virtual image, quantity of augmentedreality information included in a virtual image, or resolution of adisplay unit. For instance, the bigger the size of the display areacompared to the size of the display unit, the less quantity of theaugmented reality information included in the virtual image, and thehigher the resolution of the display unit, the greater the secondreference speed value increases. The second reference speed can be setto various values according to a user configuration and can be set withvarious ways.

Referring to FIG. 5 c, the moving speed of the virtual image 40 on eachtime (t1, t2, and t3) is faster than the second reference speed. In thiscase, it is impossible or unnecessary for a user to sufficiently readthe virtual image 40. Hence, the portable device can stop displaying thevirtual image 40. In this case, the portable device can stop displayingthe virtual image in a manner of gradually fading out the virtual image.

FIG. 5 a to FIG. 5 c depicts a case that the moving speed of the virtualimage is a constant speed, by which the present specification may benon-limited. The moving speed of the virtual image 40 can continuouslychange. Hence, the portable device can obtain the moving speed of thevirtual image 40 in real time and can perform the aforementioned dataprocessing according to the moving speed in real time. And, the portabledevice obtains the moving speed of the virtual image 40 with apredetermined time interval and can perform the aforementioned dataprocessing according to the moving speed with the predetermined timeinterval.

Although FIG. 5 a to FIG. 5 c is explained based on the moving speed ofthe virtual image 40, if the moving speed of the marker object 30 isobtained, an identical explanation can be applied as well. Inparticular, if the moving speed of the marker object 30 is slower thanthe first reference speed, the portable device moves the virtual image40 in accordance with a position change of the marker object 30 and cancontinuously display the virtual image 40 in a position adjacent to themarker object 30. And, if the moving speed of the marker object 30 isfaster than the first reference speed and slower than the secondreference speed, the portable device can lower the moving speed of thevirtual image 40 less than the first reference speed. And, if the movingspeed of the marker object 30 is faster than the second reference speed,the portable device can stop displaying the virtual image 40.

Referring to FIG. 5 d, the moving speed of the virtual image 40 on eachtime (t1 and t2) is faster than the first reference speed, movingacceleration of the virtual image 40 on a time (t2) is negativeacceleration, and the moving speed of the virtual image 40 on each time(t3 and t4) is slower than the first reference speed.

If the moving speed of the virtual image 40 is faster than the firstreference speed and the moving acceleration of the virtual image 40 isnegative acceleration as shown on the time (t2), the portable devicedetermines whether the moving speed of the virtual image is lowered lessthan the first reference speed based on a magnitude of the accelerationand can display the virtual image 40 according to the determined result.For instance, when the moving acceleration of the virtual image 40 isnegative acceleration, which is greater than a predetermined magnitude,since the moving speed of the virtual image will become slower than thefirst reference speed in a short time although the moving speed of thevirtual image 40 on a current timing point is faster than the firstreference speed, the portable device may not lower the moving speed ofthe virtual image 40.

In this case, the predetermined magnitude can be configured based on atleast one of a size ratio of a display area and a virtual image,quantity of augmented reality information included in a virtual image,or resolution of a display unit. The predetermined size can be set tovarious values according to a user configuration and can be set withvarious ways.

Unlike FIG. 5 d, the moving speed of the virtual image 40 and the movingacceleration of the virtual image 40 can continuously change. Hence, theportable device obtains the moving speed of the virtual image 40 and themoving acceleration of the virtual image 40 in real time and can performthe aforementioned data processing according to the moving speed and themoving acceleration in real time. And, the portable device obtains themoving speed of the virtual image 40 and the moving acceleration of thevirtual image 40 with a predetermined time interval and can perform theaforementioned data processing according to the moving speed of thevirtual image 40 and the moving acceleration of the virtual image 40with the predetermined time interval.

Although FIG. 5 d is explained based on the moving speed of the virtualimage 40 and the moving acceleration of the virtual image 40, if themoving speed of the marker object 30 and the moving acceleration of themarker object 30 are obtained, an identical explanation can be appliedas well. In particular, if the moving speed of the marker object 30 isfaster than the first reference speed and the moving acceleration of themarker object 30 is negative acceleration, the portable devicedetermines whether the moving speed of the virtual image 30 is loweredless than the first reference speed based on a magnitude of theacceleration and can display the virtual image 40 according to thedetermined result. For instance, if the moving acceleration of themarker object 30 has negative acceleration, which is greater than apredetermined magnitude, since the moving speed of the virtual imagewill become slower than the first reference speed in a short timealthough the moving speed of the virtual image 40 on a current timingpoint is faster than the first reference speed, the portable device maynot lower the moving speed of the virtual image 40.

FIG. 6 is a diagram of a portable device displaying a virtual image inaccordance with duration of which the marker object is detected or asize of which the marker object is detected according to one embodiment.FIG. 6 shows a virtual image displayed in the display unit on each time(t1, t2, and t3) and an interval between each of the times is identicalto each other.

FIG. 6 a is an embodiment of the portable device displaying a virtualimage when the marker object is no longer detected and the duration isshorter than predetermined duration and FIG. 6 b is an embodiment of theportable device displaying a virtual image when the size is smaller thana predetermined size.

Referring to FIG. 6 a, although the moving speed of the virtual image 40or the moving speed of the marker object 30 on each time (t1, t2, andt3) is slower than the first reference speed, the marker object is nolonger detected and the duration is shorter than predetermined duration.In this case, the portable device can further display the virtual image40 for a predetermined period after the marker object is no longerdetected. In this case, the predetermined period can be set to variousvalues according to a user configuration and can be set with variousways.

According to one embodiment, the portable device can display the virtualimage 40 as a static state in a position where the detecting of themarker object 30 is ended for a predetermined period after a time (t2)on which the detecting of the marker object 30 is ended as shown on time(t3). According to a different embodiment, the portable device changes agraphic user interface and can display the virtual image 40 inaccordance with the changed graphic user interface. Regarding this, itshall be described in detail in FIG. 7.

In this case, the predetermined duration indicates a minimum durationrequired to sufficiently read a moving virtual image. In particular,although the moving speed of the virtual image 40 is slower than thefirst reference speed, if the duration is shorter than the predeterminedduration, a user cannot sufficiently read the virtual image.

The predetermined duration can be configured based on at least one of asize ratio of an angle of view area and a marker object, quantity ofaugmented reality information included in a virtual image, or resolutionof a display unit. For instance, the bigger the size of the angle ofview area compared to the size of the display unit, the less quantity ofthe augmented reality information included in the virtual image, and thehigher the resolution of the display unit, the greater the predeterminedduration value increases. The predetermined duration can be set tovarious values according to a user configuration and can be set withvarious ways.

Referring to FIG. 6 b, although the moving speed of the virtual image 40or the moving speed of the marker image 30 is slower than the firstreference speed on each time (t1, t2, t3, and t4), the size of themarker object 30 is smaller than the predetermined size. In this case,the portable device can stop displaying the virtual image 40.

In this case, the predetermined size indicates a minimum size requiredto display a virtual image. In particular, although the moving speed ofthe virtual image 40 is slower than the first reference speed, if thesize of the marker object 30 is smaller than the predetermined size, itmay correspond to a case that the marker object 30 is apart from avision of a user as far as not necessary to display the virtual image 40for the detected marker object 30.

The predetermined size can be configured based on at least one of anangle of view area and a marker object, quantity of augmented realityinformation included in a virtual image, or resolution of a displayunit. For instance, the bigger the size of the angle of view areacompared to the size of the display unit, the less quantity of theaugmented reality information included in the virtual image, and thehigher the resolution of the display unit, the greater the predeterminedsize value increases. The predetermined size can be set to variousvalues according to a user configuration and can be set with variousways.

Although it is assumed that the moving speed of the virtual image 40 orthe moving speed of the marker object 30 is slower than the firstreference speed in FIG. 6 a and FIG. 6 b, in case that the moving speedof the virtual image 40 is faster than the first reference speed, anidentical explanation can be applied as well.

FIG. 7 is a diagram of a portable device displaying a virtual image witha changed displaying method in case of lowering a moving speed of thevirtual image according to one embodiment.

Referring to FIG. 7 a, when the moving speed of the virtual image 40 islowered less than the first reference speed, the portable device obtainsa virtual marker object 50 corresponding to the marker object 30 usingthe processor. The portable device can display the obtained virtualmarker object 50 in the display unit 130-1 together with the virtualimage 40.

In this case, the portable device can obtain the virtual marker object50 from an internal storage unit or an external storage unit. In thiscase, the virtual marker object 50 means a virtual object correspondingto the marker object 30. In this case, a shape, a color, transparency, aposition, and the like of the virtual marker object can be variously setaccording to such various methods as a user input, a pre-stored method,and the like.

As mentioned in the foregoing description, when the portable devicedisplays the virtual marker object 50 together with the virtual image40, although the marker object 30 deviates from a vision of a user, theportable device can inform the user of the marker object 30corresponding to the virtual image 40.

And, the portable device can further display an indicator indicating aconnection between the virtual marker object and the correspondingmarker object.

Referring to FIG. 7 b, when the moving speed of the virtual image 40 islowered less than the first reference speed, the portable device canfurther display the indicator 60 indicating the connection between thevirtual image 40 and the corresponding marker object 30 in the displayunit 130-2 using the processor. In this case, a shape, a color, aposition, and the like of the indicator can be variously set accordingto such various methods as a user input, a pre-stored method, and thelike.

As mentioned earlier, in case that the portable device further displaysthe indicator 60, although a position of the marker object 30 is faraway from a position of the virtual image 40, the portable device caninform a user of the marker object 30 corresponding to the virtual image40.

Referring to FIG. 7 c, when the moving speed of the virtual image 40 islowered less than the first reference speed, the portable device candisplay the virtual image 40 in a simple display mode in the displayunit 130-3 using the processor. In this case, as mentioned earlier, thesimple display mode is a display mode configured to include a part ofselected augmented reality information in a display object only amongthe augmented reality informations related to the marker object includedin the virtual image.

In this case, the processor can select a part of the augmented realityinformation only among the augmented reality informations according to apredefined priority. For instance, if the augmented reality informationincludes a bus line, a position of a next bus, information on anallocation interval and a priority is determined in an order of the busline, the position of the next bus, and the allocation interval, theprocessor can display the augmented reality information in a manner ofselecting only the augmented reality information on the bus line havinga first priority and on the position of the next bus having a secondpriority. In this case, the priority can be variously set according tosuch various methods as a user input, a predetermined method, and thelike.

As mentioned earlier, in case that the portable device displays avirtual image including less quantity of augmented reality information,although a moving speed of the virtual image is fast, the portabledevice can enhance the readability of a user for the virtual image.

Referring to FIG. 7 d, when the moving speed is lowered less than thefirst reference speed, the processor changes a graphic user interfaceformat and can display a virtual image 40 in the display unit 130-4 inaccordance with the changed graphic user interface format. Although FIG.7 d depicts the virtual image 40 displaying the virtual image 40 in amanner of changing its graphic user interface format for a displayedcolor of the virtual image, as mentioned earlier, the graphic userinterface format can include a display position, a display size, adisplay color, or a configuration value for transparency of the virtualimage 40. In this case, the graphic user interface format can bevariously set according to such various methods as a user input, apre-stored method, and the like.

In case that the portable device displays a virtual image by such ascheme of reducing a size of the virtual image and the like in a mannerof changing the graphic user interface format, although a moving speedof the virtual image is fast, the portable device can enhance thereadability of a user for the virtual image.

Although FIG. 7 depicts embodiments for a case of lowering the movingspeed of the virtual image, by which the present specification may benon-limited. In case that the virtual image is displayed as a staticstate, the identical embodiment can be applied as well.

FIG. 8 is a flowchart of a method of controlling a portable deviceaccording to one embodiment.

First of all, the portable device can capture an image in front of adevice [S810]. As mentioned earlier in FIG. 2, the portable devicecaptures the image in front of the device within an angle of view via acamera unit and can deliver the captured image to a processor. In thiscase, the camera unit can capture the image in front of the devicecontiguously or with a specific time interval.

And, the portable device can detect a marker object from the capturedimage [S820].

As mentioned earlier in FIG. 2, the portable device can detect themarker object from the image captured by the camera unit. And, theportable device can detect the marker object from a data correspondingto the image displayed in a display unit.

And, in case that the marker object is detected, the portable deviceobtains a virtual image information on a virtual image related to themarker object and can display the virtual image based on a position ofthe marker object [S830].

As mentioned earlier in FIG. 2, the portable device can obtain thevirtual image information from an internal storage unit or an externalstorage unit. In this case, the virtual image information can include atleast one of augmented reality information related to the marker object,information on a display mode, or information on a graphic userinterface format. Detailed contents are described in the foregoingdescription. As mentioned earlier in FIG. 3, the portable device obtainsa virtual image for a selected marker object or a specific marker objectpositioned at a gaze location of a user and can display the virtualimage corresponding to the specific marker object only.

And, the portable device can detect a position change of the markerobject from the captured image [S840]. As mentioned earlier in FIG. 4,the portable device can detect an absolute position change of the markerobject or a relative position change between the device and the markerobject from the image captured by the camera unit or the datacorresponding to the image displayed in the display unit. And, theportable device senses a position and a direction of the marker objectusing a sensor unit. The portable device can judge in a manner ofdistinguishing the absolute position change of the marker object fromthe relative position change of the marker object using a sensed resultand can perform a data processing corresponding to each case.

And, when the position change of the marker object is detected, theportable device moves the virtual image according to the position changeand can obtain a moving speed of the virtual image [S850].

As mentioned earlier in FIG. 5, the portable device moves the virtualimage according to the position change of the marker object and cancontinuously display the virtual image in a position adjacent to themarker object. And, the portable device can further obtain the movingspeed of the marker object. Moreover, the portable device can furtherobtain acceleration of the virtual image and acceleration of the markerobject.

And, the portable device 100 determine whether the moving speed of thevirtual image is faster than a first reference speed [S860].

As mentioned earlier in FIG. 5, the portable device can determinewhether the moving speed of the virtual image is faster than a secondreference speed. And, the portable device can determine whether themoving speed of the marker object is faster than the first referencespeed or the second reference speed. In this case, the first referencespeed is a predetermined speed. The first reference speed indicates aminimum speed required to read a moving virtual image. The secondreference speed is a predetermined speed. The second reference speedindicates a maximum speed required to read a moving virtual image 40.The second reference speed is faster than the first reference speed.Detailed contents are described in the foregoing description.

And, when the moving speed of the virtual image is faster than the firstreference speed, the portable device 100 can lower the moving speed ofthe virtual image less than the first reference speed.

Regarding this, it is identical to the contents described in FIG. 6. Asmentioned earlier in FIG. 7, when the moving speed of the virtual imageis lowered, the portable device can further display a virtual markerobject or an indicator. And, when moving speed of the virtual image islowered, the portable device can change the display mode or the graphicuser interface format and display the virtual image in accordance withthe changed display mode or the changed graphic user interface format.By using the aforementioned display method, readability of a user forthe virtual image can be enhanced.

FIG. 9 is a flowchart of a method of controlling a portable deviceaccording to a different embodiment.

First of all, the portable device can capture an image in front of adevice [S910]. And, the portable device can detect a marker object fromthe captured image [S920]. And, in case that the marker object isdetected, the portable device obtains virtual image information on avirtual image related to the marker object and can display the virtualimage based on a position of the marker object [S930]. And, the portabledevice can detect a position change of the marker object from thecaptured image [S940]. Explanation on S910 to S940 is omitted since itis identical to the contents of S810 to S840 mentioned earlier in FIG.8.

And, when the position change of the marker object is detected, theportable device moves the virtual image according to the position changeand can obtain a moving speed of the virtual image or a moving speed ofthe marker object [S950]. In particular, as mentioned earlier in FIG. 5,the portable device moves the virtual image according to the positionchange of the marker object and can continuously display the virtualimage in a position adjacent to the marker object. And, the portabledevice can obtain the moving speed of the virtual image or the movingspeed of the marker object. Moreover, the portable device can furtherobtain acceleration of the virtual image or acceleration of the markerobject.

And, the portable device 100 can determine whether the moving speed ofthe virtual image or the moving speed of the marker object is fasterthan a first reference speed [S960]. Moreover, as mentioned earlier inFIG. 5, the portable device can determine whether the moving speed ofthe virtual image or the moving speed of the marker object is fasterthan a second reference speed. In this case, the first reference speedis a predetermined speed. The first reference speed indicates a minimumspeed required to read a moving virtual image. The second referencespeed is a predetermined speed. The second reference speed indicates amaximum speed required to read a moving virtual image 40. Detailedcontents are described in the foregoing description.

And, when the moving speed of the virtual image or the moving speed ofthe marker object is faster than the first reference speed, the portabledevice can lower the moving speed of the virtual image less than thefirst reference speed [S970]. Regarding this, it is identical to thecontents described in FIG. 8.

Moreover, for clarity of explanation, each diagram is explained in amanner of being divided. Yet, it is possible to design a new embodimentto implement the new embodiment by combining the embodiments, which aredescribed in each of the diagrams. And, according to the necessity ofthose skilled in the art, designing a recording media readable by thecomputer, which has recorded a program for executing the previouslyexplained embodiments, also belongs to a scope of a right.

A portable device according to the present specification and acontrolling method therefor may not limitedly apply to the compositionand method of the aforementioned embodiments. The aforementionedembodiments may be configured in a manner of being selectively combinedthe whole of the embodiments or a part of the embodiments to achievevarious modifications.

Meanwhile, a portable device according to the present specification anda method of controlling therefor can be implemented with a code readableby a processor in a recording media readable by the processor installedin a network device. The recording media readable by the processor mayinclude all kinds of recording devices for storing data capable of beingread by the processor. The examples of the recording media readable bythe processor may include a ROM, a RAM, a magnetic tape, a floppy disc,an optical data storing device and the like. And, implementing in a formof a carrier wave such as a transmission via the internet and the likeis also included. The recording media readable by the processor aredistributed to the computer systems connected by a network and codesreadable by the processor can be stored and executed in a manner ofbeing distributed.

While the present specification has been described and illustratedherein with reference to the preferred embodiments and diagrams thereof,the present specification may be non-limited to the aforementionedembodiments and it will be apparent to those skilled in the art thatvarious modifications and variations can be made therein withoutdeparting from the spirit and scope of the present specification. Thus,it is intended that the present specification covers the modificationsand variations of this invention that come within the scope of theappended claims and their equivalents.

And, both an apparatus invention and a method invention are explained inthe present specification and the explanation on the both of theinventions can be complementally applied, if necessary.

What is claimed is:
 1. A portable device, comprising: a camera unitconfigured to capture an image in front of the portable device; adisplay unit configured to display a virtual image; and a processorconfigured to control the camera unit and the display unit, theprocessor further configured to: detect a marker object from the image,display the virtual image corresponding to the marker object based on aposition of the marker object when the marker object is detected, detecta position change of the marker object, move the virtual image accordingto the position change when the position change is detected, obtain afirst moving speed of the virtual image or a second moving speed of themarker object, and when the first moving speed or the second movingspeed is faster than a first reference speed, lower the first movingspeed to less than the first reference speed.
 2. The portable device ofclaim 1, wherein the processor is further configured to stop displayingthe virtual image when the first moving speed or the second moving speedis faster than a second reference speed, wherein the second referencespeed is faster than the first reference speed.
 3. The portable deviceof claim 1, wherein the processor is further configured to display avirtual object corresponding to the marker object together with thevirtual image when the first moving speed is lowered and less than thefirst reference speed.
 4. The portable device of claim 1, wherein theprocessor is further configured to additionally display an indicatorindicating a connection between the virtual image and the marker objectwhen the first moving speed is lowered and less than the first referencespeed.
 5. The portable device of claim 1, wherein the information on thevirtual image comprises at least one of augmented reality informationrelated to the marker object, display mode information, and graphic userinterface format information.
 6. The portable device of claim 1, whereinthe processor is further configured to display the virtual image with asimple display mode when the first moving speed is lowered and less thanthe first reference speed.
 7. The portable device of claim 6, whereinthe simple display mode corresponds to a display mode only displayingselected augmented reality information of augmented reality informationcontained in the virtual image.
 8. The portable device of claim 1,wherein the processor is further configured to: obtain information onduration of which the marker object is detected when the position changeis detected, and display the virtual image for a predetermined periodwhen the marker object is no longer detected and the duration is shorterthan predetermined duration.
 9. The portable device of claim 8, whereinthe processor is further configured to display the virtual image as astationary state in a position where the marker object is lastlydetected.
 10. The portable device of claim 1, wherein the processor isfurther configured to: obtain information on a size of which the markerobject is detected when the position change is detected, and stopdisplaying the virtual image when the size is smaller than apredetermined size.
 11. The portable device of claim 1, furthercomprising a sensor unit configured to sense a user gaze and deliver asensed result to the processor, Wherein the processor is furtherconfigured to: detect a gaze location where the user's gaze is on when aplurality of marker objects is detected, and select at least one markerobject from the plurality of marker objects based on the gaze location.12. The portable device of claim 11, wherein the processor is furtherconfigured to select at least one marker object, which is positionedwithin a predetermined distance from the gaze location.
 13. The portabledevice of claim 1, wherein the processor is further configured to:change a graphic user interface format when the first moving speed islowered and less than the first reference speed, and display the virtualimage according to the changed graphic user interface format.
 14. Theportable device of claim 13, wherein the graphic user interface formatcomprises at least one of a display position, a display size, a displayshape, a display color, or a display transparency of the virtual image.15. The portable device of claim 1, wherein the portable devicecomprises a head mounted display (HMD).
 16. The portable device of claim15, wherein the display unit consists of an optical see-through displaypanel.
 17. The portable device of claim 1, wherein when the first movingspeed or the second moving speed is faster than the first referencespeed, the processor is further configured to obtain acceleration of thevirtual image or acceleration of the marker object, and determinewhether the first moving speed is lowered and less than the firstreference speed based on the obtained acceleration.
 18. The portabledevice of claim 17, wherein the processor is further configured not tolower the first moving speed less than the first reference speed whenthe obtained acceleration is negative acceleration, which is greaterthan a predetermined magnitude.
 19. A method of controlling a portabledevice, comprising the steps of: capturing an image in front of theportable device; detecting a marker object from the image; when themarker object is detected, obtaining an information on the virtual imagecorresponding to the marker object and displaying the virtual imagebased on a position of the marker object; detecting a position change ofthe marker object; when the position change is detected, moving thevirtual image according to the position change and obtaining a movingspeed of the virtual image or a moving speed of the marker object; andwhen the moving speed of the virtual image or the moving speed of themarker object is faster than a first reference speed, lowering themoving speed of the virtual image to less than the first referencespeed.
 20. The method of claim 19, when the moving speed of the virtualimage or the moving speed of the marker object is faster than a secondreference speed, further comprising the step of stopping displaying thevirtual image, wherein the second reference speed is faster than thefirst reference speed.